Memory Cache Management for Graphics Processing

ABSTRACT

Systems, methods, and computer readable media to manage memory cache for graphics processing are described. A processor creates a resource group for a plurality of graphics application program interface (API) resources. The processor subsequently encodes a set command that references the resource group within a command buffer and assigns a data set identifier (DSID) to the resource group. The processor also encodes a write command within the command buffer that causes the graphics processor to write data within a cache line and mark the written cache line with the DSID, a read command that causes the graphics processor to read data written into the resource group, and a drop command that causes the graphics processor to notify the memory cache to drop, without flushing to memory, data stored within the cache line.

BACKGROUND

This disclosure relates generally to the field of graphics processing.More particularly, but not by way of limitation, this disclosure relatesto having a graphics processor, such as a graphics processing unit(GPU), managing memory cache by tagging and dropping data setidentifiers (DSIDs) associated with cache lines.

Computers, mobile devices, and other computing systems typically have atleast one programmable processor, such as a central processing unit(CPU) and other programmable processors specialized for performingcertain processes or functions (e.g., graphics processing). Examples ofa programmable processor specialized to perform graphics processingoperations include, but are not limited to, a GPU, a digital signalprocessor (DSP), a field programmable gate array (FPGA), and/or a CPUemulating a GPU. GPUs, in particular, comprise multiple execution cores(also referred to as shader cores) designed to execute the sameinstruction on parallel data streams, making them more effective thangeneral-purpose processors for operations that process large blocks ofdata in parallel. For instance, a CPU functions as a host and hands-offspecialized parallel tasks to the GPUs. Specifically, a CPU can executean application stored in system memory that includes graphics dataassociated with a video frame. Rather than processing the graphics data,the CPU forwards the graphics data to the GPU for processing; thereby,freeing the CPU to perform other tasks concurrently with the GPU'sprocessing of the graphics data.

GPU processing, such as render-to-texture passes, often write and readfrom memory cache to improve performance and power saving. For instance,a render-to-texture pass renders a frame to a texture resource that canbe later re-passed to a shader for further processing. By doing so, theGPU could be writing to and/or reading from the texture resource beforethe GPU is done utilizing the texture resource. Having the textureresource accessible within memory cache during this time period couldimprove GPU performance. However, the memory cache may be unaware ofwhen a GPU is done utilizing the texture resource. In certain instances,while the GPU is still using the texture resource, the memory cache maypersist data out to system memory (e.g., DRAM) causing a reduction inGPU performance.

SUMMARY

In one implementation, a method is disclosed to associate DSIDs forwrite commands and subsequently drop written data (if any) from thecache using the DSIDs. The example method creates a resource group for aplurality of graphics application program interface (API) resources,where each graphics API resource corresponds to a memory allocation forstoring data that is accessible to a graphics processor. The examplemethod encodes a set command that references the resource group within acommand buffer. The set command assigns a DSID to the resource group. Awrite command within the command buffer causes the graphics processor towrite data within a cache line. The write command is also able to causethe written cache line to be associated with the DSID. A read commandcauses the graphics processor to read data written into the resourcegroup. A drop command causes the graphics processor to notify the memorycache to drop, without flushing to memory, data stored within thewritten cache line associated with the DSID. The processor then commitsone or more command buffers that include the set command, the writecommand, the read command, and the drop command for execution on thegraphics processor.

In another implementation, a system for utilizing write commands anddrop commands, where the system comprises memory and a processoroperable to interact with the memory. The processor is able to encode aset command that references a resource group within a command buffer.The set command assigns a DSID to the resource group that includes aplurality of graphics API resources. The processor is also able toencode a write command within the command buffer that causes thegraphics processor to write data within the cache line. The writecommand also causes the DSID to be marked to the written cache line. Adrop command is encoded to cause the graphics processor to notify thememory cache to delete, without evicting to memory, data stored withinthe written cache line marked with the DSID. The processor then commitsone or more command buffers that include the set command, the writecommand, and the drop command for execution on the graphics processor.

In yet another implementation, a system for associating DSIDs for writecommands and subsequently drop written data (if any) from the cachelines using the DSIDs. The system comprises memory and a graphicsprocessor operable to interact with the memory. The graphics processorobtains from one or more command buffers a write command and a dropcommand, both of which are associated with a DSID. The graphicsprocessor executes the write command to write data within the cache lineto generate a dirty cache line in a memory cache. The DSID correspondsto a resource group that includes a plurality of graphics API resourcesand the write command marks the DSID to the dirty cache line.Afterwards, the graphics processor executes, after the write command,the drop command to notify the memory cache to drop the written datawithin the dirty cache line. The data is dropped without flushing thedirty cache line to main memory.

In yet another implementation, each of the above described methods, andvariation thereof, may be implemented as a series of computer executableinstructions. Such instructions may use any one or more convenientprogramming language. Such instructions may be collected into enginesand/or programs and stored in any media that is readable and executableby a computer system or other programmable control device.

BRIEF DESCRIPTION OF THE DRAWINGS

While certain implementations will be described in connection with theillustrative implementations shown herein, this disclosure is notlimited to those implementations. On the contrary, all alternatives,modifications, and equivalents are included within the spirit and scopeof this disclosure as defined by the claims. In the drawings, which arenot to scale, the same reference numerals are used throughout thedescription and in the drawing figures for components and elementshaving the same structure, and primed reference numerals are used forcomponents and elements having a similar function and construction tothose components and elements having the same unprimed referencenumerals.

FIG. 1 is a diagram of a graphics processing path where implementationsof the present disclosure may operate.

FIG. 2 is a block diagram of a system where implementations of thepresent disclosure may operate.

FIG. 3 is a block diagram of a memory virtualization architecture formanaging memory cache when assigning, associating, and dropping a DSIDfor a resource group.

FIG. 4 is an implementation of a command buffer that includes DSIDcommands that reference a created resource group.

FIG. 5 is an implementation of referencing a created resource groupacross multiple command buffers within a command queue.

FIG. 6 is an implementation of referencing a created resource groupacross multiple command queues.

FIG. 7 depicts a flowchart illustrating a graphics processing operationfor managing memory cache for graphics processing.

FIG. 8 is a block diagram of a computing system where implementations ofthe present disclosure may operate.

FIG. 9 is a block diagram of an implementation of a software layer andarchitecture where implementations of the present disclosure mayoperate.

FIG. 10 is a block diagram of another implementation of a software layerand architecture where implementations of the present disclosure mayoperate.

DETAILED DESCRIPTION

This disclosure includes various example implementations that assign aresource group to a DSID, associate the DSID to a cache line whenwriting to the resource group, and subsequently drop the DSID when anapplication no longer needs to access content in the resource group. Inone implementation, a graphics API (e.g., OpenGL®, Direct3D®, or Metal®(OPENGL is a registered trademark of Silicon Graphics, Inc.; DIRECT3D isa registered trademark of Microsoft Corporation; and METAL is aregistered trademark of Apple Inc.)) allows a developer and/orapplication to create a resource group that includes one or moreresources (e.g., buffers and textures). The graphics API also allows acentral processing unit (CPU) to generate one or more set commandswithin a command buffer to acquire a DSID for the created resourcegroup. The command buffer could also include one or more write commandsthat mark and/or update a specific cache line within memory cache withthe DSID when writing data into the resource group, one or more readcommands for reading data from the resource group, and/or one or moredrop commands for dropping the specific cache line associated with theDSID. After the CPU presents and commits the command buffer to the GPUfor execution, the graphics driver schedules the set commands, writecommands, read commands, and/or drop commands within the committedcommand buffer for the GPU to execute. When the GPU executes the dropcommand associated with the DSID, the GPU provides a drop hint to thememory cache for dropping data stored in the cache line that correspondsto the DSID without flushing the contents to system memory. The graphicsAPI also allows a CPU to encode commands that assign and/or drop DSIDsfor resource groups across different command buffers and/or acrossdifferent command queues.

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the inventive concept. As part of this description,some of this disclosure's drawings represent structures and devices inblock diagram form in order to avoid obscuring the disclosed principles.In the interest of clarity, not all features of an actual implementationare described. Moreover, the language used in this disclosure has beenprincipally selected for readability and instructional purposes, and maynot have been selected to delineate or circumscribe the inventivesubject matter, resort to the claims being necessary to determine suchinventive subject matter. Reference in this disclosure to “oneimplementation” or to “an implementation” means that a particularfeature, structure, or characteristic described in connection with theimplementation is included in at least one implementation of thedisclosure, and multiple references to “one implementation” or “animplementation” should not be understood as necessarily all referring tothe same implementation.

The terms “a,” “an,” and “the” are not intended to refer to a singularentity unless explicitly so defined, but include the general class ofwhich a specific example may be used for illustration. The use of theterms “a” or “an” may therefore mean any number that is at least one,including “one,” “one or more,” “at least one,” and “one or more thanone.” The term “or” means any of the alternatives and any combination ofthe alternatives, including all of the alternatives, unless thealternatives are explicitly indicated as mutually exclusive. The phrase“at least one of” when combined with a list of items, means a singleitem from the list or any combination of items in the list. The phrasedoes not require all of the listed items unless explicitly so defined.

As used herein, the term “kernel” in this disclosure refers to acomputer program that is part of a core layer of an operating system(e.g., Mac OSX™) typically associated with relatively higher or thehighest security level. The “kernel” is able to perform certain tasks,such as managing hardware interaction (e.g., the use of hardwaredrivers) and handling interrupts for the operating system. To preventapplication programs or other processes within a user space frominterfering with the “kernel,” the code for the “kernel” is typicallyloaded into a separate and protected area of memory. Within thiscontext, the term “kernel” may be interchangeable throughout thisdisclosure with the term “operating system kernel.”

The disclosure also uses the term “compute kernel,” which has adifferent meaning and should not be confused with the term “kernel” or“operating system kernel.” In particular, the term “compute kernel”refers to a program for a graphics processor (e.g., GPU, DSP, or FPGA).In the context of graphics processing operations, programs for agraphics processor are classified as a “compute kernel” or a “shader.”The term “compute kernel” refers to a program for a graphics processorthat performs general compute operations (e.g., compute commands), andthe term “shader” refers to a program for a graphics processor thatperforms graphics operations (e.g., render commands).

As used herein, the term “command” in this disclosure refers to agraphics API command encoded within a data structure, such as commandbuffer or command list. The term “command” can refer to a render command(e.g., for draw calls) and/or a compute command (e.g., for dispatchcalls) that a graphics processor is able to execute. Examples ofcommands pertinent to this disclosure that relate to managing memorycache include a “set command” that acquires a DSID for a createdresource group, a “write command” (e.g., render command) that associatesthe DSID to a written cache line for the purpose of writing to aresource group, a “read command” (e.g., render command) that reads fromthe resource group and a “drop command” (e.g., render command) fornotifying (e.g., a drop hint) a memory cache that it is able to drop thewritten cache line associated with the DSID.

For the purposes of this disclosure, the term “processor” refers to aprogrammable hardware device that is able to process data from one ormore data sources, such as memory. One type of “processor” is ageneral-purpose processor (e.g., a CPU) that is not customized toperform specific operations (e.g., processes, calculations, functions,or tasks), and instead is built to perform general compute operations.Other types of “processors” are specialized processor customized toperform specific operations (e.g., processes, calculations, functions,or tasks). Non-limiting examples of specialized processors include GPUs,floating-point processing units (FPUs), DSPs, FPGAs,application-specific integrated circuits (ASICs), and embeddedprocessors (e.g., universal serial bus (USB) controllers).

As used herein, the term “graphics processor” refers to a specializedprocessor for performing graphics processing operations. Examples of“graphics processors” include, but are not limited to, a GPU, DSPs,FPGAs, and/or a CPU emulating a GPU. In one or more implementations,graphics processors are also able to perform non-specialized operationsthat a general-purpose processor is able to perform. As previouslypresented, examples of these general compute operations are computecommands associated with compute kernels.

As used herein, the term “resource” refers to an allocation of memoryspace for storing data that is accessible to a graphics processor, suchas a GPU, based on a graphics API. For the purpose of this disclosure,the term “resource” is synonymous and can also be referenced as“graphics API resource.” Examples of graphics API resources includebuffers and textures. Buffers represent an allocation of unformattedmemory that can contain data, such as vertex, shader, and compute statedata. Textures represents an allocation of memory for storing formattedimage data. The term “resource group” refers to a data structure thatcontains a list of resources that are logically grouped together for aninterim time period. In one implementation, the resource group is animmutable list of resources where a resource cannot be added to orremoved from the resource group once an application creates the resourcegroup.

FIG. 1 is a diagram of a graphics processing path 100 whereimplementations of the present disclosure may operate. FIG. 1illustrates an example in which the graphics processing path 100utilizes a processor resource 110 and a graphics processor resource 112.The processor resource 110 includes one or more general-purposeprocessors (e.g., CPUs), where each processor has one or more cores. Theprocessor resource 110 can also contain and/or communicate with memory,microcontrollers, and/or any other hardware resources a processor mayutilize to process commands for graphics processor resource 112 toexecute. The graphics processor resource 112 includes one or moregraphics processors (e.g., GPUs), where each graphics processor has oneor more execution cores and other computing logic for performinggraphics and/or general compute operations. Stated another way, thegraphics processor resource 112 may also encompass and/or communicatewith memory (e.g., memory cache 108), and/or other hardware resources toexecute programs, such as shaders or compute kernels. For example,graphics processor resource 112 is able to process shaders with arendering pipeline and compute kernels with a compute pipeline.

FIG. 1 illustrates that application 101 generates graphics API calls forthe purpose of encoding commands for the graphics processor resource 112to execute. To generate the graphics API calls, application 101 includescode written with a graphics API. The graphics API (e.g., Metal®)represents a published and/or standardized graphics library andframework that define functions and/or other operations that application101 is able to have with a graphics processor. For example, the graphicsAPI allows application 101 to be able to control the organization,processing, and submission of render and compute commands, as well asthe management of associated data and resources for those commands.

In one or more implementations, application 101 is a graphicsapplication that invokes the graphics API to convey a description of agraphics scene. Specifically, the user space driver 102 receivesgraphics API calls from application 101 and maps the graphics API callsto operations understood and executable by the graphics processorresource 112. For example, the user space driver 102 can translate theAPI calls into commands encoded within command buffers before beingtransferred to kernel driver 103. The translation operation may involvethe user space driver 102 compiling shaders and/or compute kernels intocommands executable by the graphics processor resource 112. The commandbuffers are then sent to the kernel driver 103 to prepare the commandbuffers for execution on the graphics processor resource 112. As anexample, the kernel driver 103 may perform memory allocation andscheduling of the command buffers to be sent to the graphics processorresource 112. For the purpose of this disclosure and to facilitate easeof description and explanation, unless otherwise specified, the userspace driver 102 and the kernel driver 103 are collectively referred toas a graphics driver.

FIG. 1 illustrates that the graphics processor firmware 104 obtainscommand buffers that processor resource 110 commits for execution. Thegraphics processor firmware 104 can perform a variety of operations tomanage the graphics processor hardware 105 that includes powering up thegraphics processor hardware 105 and/or scheduling the order of commandsthat the graphics processor hardware 105 receives for execution. Withreference to FIG. 1 as an example, the graphics processor firmware 104can be implemented by a microcontroller that executes the graphicsprocessor firmware 104. Specifically, the microcontroller could beembedded in the same package as a graphics processor within the graphicprocessor resource 112 and setup to pre-process commands for thegraphics processor. In other implementations, the microcontroller isphysically separated from the graphics processor.

After scheduling the commands, in FIG. 1, the graphics processorfirmware 104 sends command streams to the graphics processor hardware105. The graphics processor hardware 105 then executes the commandswithin the command streams according to the order the graphics processorhardware 105 receives the commands. The graphics processor hardware 105includes multiple (e.g., numerous) execution cores, and thus, canexecute a number of received commands in parallel. The graphicsprocessor hardware 105 then outputs rendered frames to frame buffer 106.In one implementation, the frame buffer 106 is a portion of memory, suchas a memory buffer, that contains a bitmap that drives display 107.Display 107 subsequently accesses the frame buffer 106 and converts(e.g., using a display controller) the rendered frame (e.g., bitmap) toa video signal for display.

In one or more implementations, the graphics processing path 100 is alsoable to support creating resource groups, assigning a DSID to thecreated resource group, associating the DSID to a cache line with awrite command, and dropping the DSID for the created resource group. InFIG. 1, application 101 may generate a graphics API call to create aresource group that logically groups resources (e.g., buffers andtextures) for a certain time period. The graphics API also allowsprocessor resource 110 to generate a set command within a command bufferto acquire and assign a DSID for the created resource group. The commandbuffer could also include a write command tagged with a DSID to writedata into the resource group, a read command to read data from theresource group, and/or a drop command for dropping the cache lineassociated with the assigned DSID.

After the processor resource 110 commits the command buffer to thegraphics processor resource 112, the graphics driver schedules the setcommand, write command, read command, and/or drop command for execution.When the graphics processor resource 112 executes the drop command thatreferences the DSID, the graphics processor hardware 105 notifies memorycache 108 that the memory cache 108 no longer needs to store thecontents within the given cache line. As a result, the memory cache 108can drop contents within the given cache line that corresponds to theDSID. When the memory cache 108 drops contents in the cache line, thememory cache 108 invalidates the contents stored in the cache line suchthat the cache line's contents are not flushed to memory that is mappedto the memory cache (e.g., main memory).

In one or more implementations, the memory cache 108 includes a cachecontroller (not shown in FIG. 1) that accesses the actual blocks ofmemory cache. The drop notification the graphics processor hardware 105sends to the memory cache 108 acts as a drop hint. The cache controllerdrops content stored in the corresponding cache line if the memory cache108 still stores the contents and has not previously evicted thecontents to system memory. In some instances, the memory cache 108 hasalready evicted contents stored in the cache line prior to receiving thedrop hint from the graphics processor hardware 105. When this occurs,the cache controller does not drop content stored in the memory cache108 in response to receiving the drop notification. For the purpose ofthis disclosure, the term “flush” can also be referred throughout thisdisclosure as “persist” or “evict.”

The user space driver 102 is configured to manage and assign the DSIDsto resource groups. DSIDs represent a set of virtual identifiers thatare eventually associated with cache lines. When the user space driver102 receives an API call to set a resource group, the user space driver102 acquires and assigns a DSID to a created resource group. Forexample, the user space driver 102 may initially have available a totalof about 64,000 DSIDs to assign to a created resource group. The userspace driver 102 acquires one of the available DSIDs (e.g., a DSID notallocated to another resource group) and assign the available DSID tothe created resource group. As shown in FIG. 1, the user space driver102 then produces the DSID associated with the resource group within theset command to the kernel driver 103.

In one or more implementations, after receiving the set command from theuser space driver 102, the kernel driver 103 assigns the DSID receivedfrom the user space driver 102 to a hardware DSID. The DSID that theuser space driver 102 assigns to the created resource group, thus, actsas a virtual identifier that the kernel driver 103 then maps to ahardware DSID. The kernel driver 103 may also maintain other mappingsbetween other hardware DSIDs and other DSIDs allocated to other resourcegroups. The kernel driver 103 may also track work (e.g., kicks)submitted to graphics processor resource 112 that utilize the hardwareDSID. The kernel driver 103 provides the hardware DSIDs and worktracking information to the graphics processor firmware 104. Thegraphics processor firmware 104 may utilize the hardware DSIDs and worktracking information to manage operations between the graphics processorhardware 105 and memory cache 108. For example, the graphics processorfirmware 104 may dictate when the graphics processor hardware 105accesses memory cache 108 for a given hardware DSID and manage when toinitiate a drop notification to the memory cache 108 for a givenhardware DSID.

After a set command assigns the created resource group to a DSID andhardware DSID, application 101 references the created resource group bytagging the DSID within other commands in the command buffer (e.g.,write command and read command). The user space driver 102 subsequentlypasses the other commands tagged with the DSID to the kernel driver 103.For write commands, the user space driver 102 may also tag an addressrange with the DSID. In one or more implementations, the address rangecould have a range start and end address that is about 128 bytes long.The user space driver 102 is responsible for ensuring that the addressranges for each DSID does not overlap. Afterwards, the kernel driver 103determines the hardware DSID that maps to the DSID and forwards thetagged commands with the hardware DSID to the graphics processorresource 112. The graphics processor resource 112 then executes thetagged commands using the hardware DSID.

When application 101 issues a write command for a resource group, one ormore cache lines in memory cache 108 that store the written content forthe resource group becomes associated with DSID and hardware DSID. Thehardware DSID represents the identifier the graphics processor hardware105 uses to generate a notification to the memory cache 108 to dropcontents within the corresponding cache line. Cache line is a term ofart understood by skilled artisans that refers to a block of memory thatis transferred to or from a memory cache 108. A cache line could have afixed data size when transferring data to or from memory cache 108. Forexample, the cache line could bet set to include multiple bytes and/orwords of data, where a whole cache line is read from or written toduring a data transfer. A hardware DSID may correspond to or be part oftags, indexes, or other address information used to identify a cacheline.

When application 101 is done utilizing a resource group, application 101may initiate a drop command for the purposes of notifying memory cache108 that contents stored in a cache line can be dropped (e.g., deletedor marked as available to be overwritten) from the memory cache 108without evicting the contents to a mapped memory location in mainmemory. After the user space driver 102 receives the drop command for agiven DSID, the user space driver 102 may include the given DSIDassociated with the drop command within a list of DSIDs that can bedropped from the memory cache 108. The user space driver 102subsequently sends the list of DSIDs to the kernel driver 103. Thekernel driver 103 and graphics processor firmware 104 manages andschedules drop operations for the list of DSIDs such that dropoperations do not occur until the graphics processor resource 112completes write and read commands associated with a given DSID. Once thegraphics processor resource 112 completes a drop operation, the userspace driver 102 and the kernel driver 103 are able to free up and reusethe given DSID and mapped hardware DSID, respectively, for new setcommands.

Although FIG. 1 illustrates a specific implementation of graphicsprocessing path 100, the disclosure is not limited to the specificimplementation illustrated in FIG. 1. For instance, graphics processingpath 100 may include other frameworks, APIs, and/or application layerservices not specifically shown in FIG. 1. As an example, application101 may have access to Core Animation to animate views and/or userinterfaces for application 101. FIG. 1 also does not illustrate all ofthe hardware resources and/or components that graphics processing path100 may utilize (e.g., power management units or memory resources, suchas system memory). Additionally or alternatively, even though FIG. 1illustrates that processor resource 110 and graphics processor resource112 are separate devices, other implementations could have the processorresource 110 and graphics processor resource 112 integrated on a singledevice (e.g., a system-on-chip). The use and discussion of FIG. 1 isonly an example to facilitate ease of description and explanation.

FIG. 2 is a block diagram of a system 200 where implementations of thepresent disclosure may operate. Specifically, system 200 is able toimplement the graphics processing path 100 shown in FIG. 1. FIG. 2illustrates that system 200 includes a processor resource 110 and agraphics processor resource 112. FIG. 2 illustrates processor threads204A and 204B. Processor thread 204A is tasked with utilizing commandencoders 206A and 206B and processor thread 204B is tasked withutilizing command encoder 206C and 206D. The command encoders 206A and206B encode commands within command buffer 208A and command encoders206C and 206D encode commands within command buffer 208B. A differentnumber of processor threads and command encoders can be included inother implementations compared to two processor threads and four commandencoders shown in the example of FIG. 2. The command encoders 206A-206Drepresents encoders that encodes commands into command buffers 208A and208B for the graphics processor resource 112 to execute. Examples ofcommand encoder types include, but are not limited to, Blit commandencoders (e.g., graphics API resource copy and graphics API resourcesynchronization commands), compute command encoders (e.g., computecommands), and render command encoders (e.g., render commands).

Command buffers 208A and 208B, which are also referred to as “commandlists,” represent data structures that store a sequence of encodedcommands for graphics processor resource 112 to execute. When one ormore graphics API calls present and commit command buffers 208A and 208Bto a graphics driver (e.g., the user space driver 102 shown FIG. 1), theprocessor resource 110 organizes the command buffers 208A and 208B intoa command queue 210. The command queue 210 organizes the order in whichcommand buffers 208 are sent to graphics processor resource 112 forexecution. Using FIG. 2 as an example, command queue 210 containscommand buffers 208C-208N, where command buffer 208C is at the top ofthe command queue 210 and is the next command buffer 208C to be sent tographics processor resource 112 for execution. When processor resource110 commits command buffers 208A and 208B for execution, the processorresource 110 is unable to encode any additional commands into commandbuffers 208A and 208B. After committing a command buffer 208, thecommand buffer becomes available to the graphics processor resource 112for execution.

The example of FIG. 2 also illustrates that processor resource 110 andgraphics processor resource 112 bilaterally communicate with a memorycontroller 202. The memory controller 202 manages the flow ofinformation to and from system memory 212 and is sometimes tasked withmaintaining system memory 212 itself (e.g., refresh or otherfunctionality depending upon the type of memory). As shown in FIG. 2, asingle memory controller 202 performs memory control for both theprocessor resource 110 and graphics processor resource 112. In anotherimplementation, the memory controller 202 includes separate memorycontrollers, one memory control for processor resource 110 and anothermemory controller for graphics processor resource 112. The memorycontroller 202 bilaterally communicates with system memory 212, whichmay be divided into processor resource memory 214 and graphics processorresource memory 216. Some implementations of system memory 212 usephysically or logically independent memory for each of the processorresource 110 and graphics processor resource 112, while otherimplementations call for sharing system memory 212 on a physical orlogical basis.

Using FIG. 2 as an example, processor resource 110 is able to generate aset command within a command buffer 208 (e.g., 208C) to acquire a DSIDfor a created resource group. The same command buffer 208 (e.g., 208C)could also include a write command that is tagged with the DSID forwriting data into the resource group, read command for reading data fromthe resource group, and/or a drop command for dropping the cache lineassociated with the DSID. In another implementation, different commandbuffers 208 (e.g., 208C, 208D, and/or 208E) could include a writecommand, read command, and drop command that reference the same DSID.After processor resource 110 commits the command buffers 208 to graphicsprocessor resource 112, the graphics driver schedules the DSID commandsfor execution on the graphics processor resource 112.

When the graphics processor resource 112 executes the drop commandassociated with the DSID, the graphics processor hardware 105 notifiesmemory cache 108 that it is able to invalidate the contents stored inthe corresponding cache line such that the cache line contents do notneed to be evicted out to system memory 212. The graphics processorhardware 105 does not guarantee the contents within the memory cache areactually dropped, but instead provides a drop hint to memory cache 108that the application is done with the contents identified by the DSID.For example, after memory cache 108 receives the drop hint from thegraphics processor hardware 105, if memory cache 108 still contains thecontent that corresponds to the DSID, then memory cache 108 drops thecache line (e.g., a dirty cache line) without flushing the contents tosystem memory 212. In some instances, memory cache 108 is unable dropthe cache line that corresponds to the DSID when memory cache 108 hasalready persisted the cache line to system memory 212, or specifically,graphics processor resource memory 216. A “dirty cache line” refers to amodified cache line that has not been modified in a memory mapped to thememory cache (e.g., main memory or system memory 212). Stated anotherway, the data stored in the cache line differs from its counterpartstored in system memory 212.

Having a graphics API that supports creating a resource group,allocating a DSID to the resource group, associating a DSID with writecommands that reference the resource group, and dropping the DSIDreferencing the resource group could provide performance and powerbenefits. In particular, exposing memory cache management to anapplication for graphics processing could reduce bandwidth usage. Forinstance, having the graphics processor continuously access memory cache108 rather than system memory 212 to perform render-to-texture passesreduces that bandwidth usage for running an application. An overallreduction in bandwidth usage translates to increases in performance forbandwidth bound type operations and reduces power consumption. Issuingdrop hints to memory cache 108 to delete cache content prior topersisting the content into system memory also provides additional powersavings. Consumption of less power also produces less heat from system200. In one implementation, system 200 is subject to thermal mitigationoperations that reduce frequencies and power to system 200. By do so,the thermal mitigation operations causes system 200 to enter a reducedperformance state.

Although FIG. 2 illustrates a specific implementation of a system 200 toassociate DSIDs for write commands and subsequently drop written data(if any) from memory cache 108 using the DSIDs, the disclosure is notlimited to the specific implementation illustrated in FIG. 2. Forinstance, even though FIG. 2 illustrates a single command queue 210;persons of ordinary skill in the art are aware that command buffers 208can be placed into other command queues 210 not shown in FIG. 2. The useand discussion of FIG. 2 is only an example to facilitate ease ofdescription and explanation.

FIG. 3 is a block diagram of a memory virtualization architecture 300for managing memory cache when assigning, tagging, and dropping a DSIDfor a resource group. FIG. 3 illustrates that application 101 creates aresource group 302 that includes one or more resources. One or moreresources within resource group 302 may also belong to other resourcegroups 302 the application 101 previously created. After creatingresource group 302, application 101 may send a graphics API call to setresource group 302 to user space driver 102. In response to the graphicsAPI call, user space driver 102 assigns a DSID 304 to resource group302. For example, the user space driver 102 may initially have availablea total of about 64,000 DSIDs to assign to resource group 302. Based onthe graphics API call to set resource group 302, the user space driver102 assigns one of the available DSID (e.g., a DSID not allocated toanother resource group) to the resource group 302.

The kernel driver 103 then maps the DSID 304 to a hardware DSID 306. Bydoing so, DSID 304 acts as a virtual identifier that the kernel driver103 then maps to a hardware DSID 306. Having a memory virtualizationarchitecture 300 that maps DSID 304 to hardware DSID 306 allows thegraphics API architecture of managing and assigning DSID 304 to beseparate and independent from the hardware architecture for managing andassigning hardware DSID 306. For example, if the architecture formanaging and assigning hardware DSID 306 for the graphics processorchanges because of redesigned hardware in the graphics processor, nosubstantial revisions may need to be made the graphics API architecture.

As previously disclosed, the kernel driver 103 may also maintain othermappings between other hardware DSIDs 306 and other DSIDs 304 allocatedto other resource groups 302. The graphics processor firmware 104 mayreceive the hardware DSID from kernel driver 103 to manage operationsbetween a graphics processor and memory cache 108. As an example, thegraphics processor firmware 104 may determine when to initiate a dropnotification to the memory cache 108 for a given hardware DSID. Based onthe graphics processor firmware 104, the graphics processor is able tocommunicate with the memory cache 108 to access cache line 308associated with the hardware DSID 306. As an example, the graphicsprocessor may read, write, and/or drop contents from a cache lineassociated with the hardware DSID 306. Recall that a hardware DSID 306may correspond to or be part of tags, indexes, or other addressinformation used to identify a cache line.

After executing a set command, the DSID 304 and hardware DSID 306 maybecome associated with cache line 308 based on a write command forresource group 302. Application 101 may generate a write command thatwrites data into resource group 302. If the write command causes atleast some of the data for resource group 302 to be written within cacheline 308, the write command is also able to associate the DSID 304 andhardware DSID 306 to the written cache line 308. Stated another way,DSID 304 and hardware DSID 306 are used to mark cache line 308 updatedwith the write command for resource group 302. In one or moreimplementations, the DSID 304 and hardware DSID 306 can be associatedwith more than one relevant cache line 308. Afterwards, DSID 304 andhardware DSID 306 can be later used to drop any cache lines associatedwith the two identifiers.

In one or more implementations, the user space driver 102 and/or thekernel driver 103 may allocate a default DSID 304 and/or a defaulthardware DSID 306, respectively (e.g., a value zero for DSID 304 and/orhardware DSID 306). A default DSID 304 and default hardware DSID 306each represent an identifier that instructs a graphics processor totreat resource group 302 as a normal cache line within memory cache 108.In other words, resource groups 302 that are assigned with the defaultDSID 304 and/or default hardware DSID 306 do not benefit from the tagand drop operations previously discussed with reference to FIGS. 1 and2. One situation where the use of default DSID 304 and/or defaulthardware DSID 306 arises when the user space driver 102 and/or thekernel driver 103 do not have any available DSID 304 and/or hardwareDSID 306 to assign after receiving the set command. Additionally oralternatively, a default hardware DSID 306 may be useful when one ormore resources belong to multiple resource groups 302. Within the memoryvirtualization architecture 300, application 101 in some instances mayinadvertently set two resource groups 302 that have at least one commonresource. Rather than having a graphics processor assign a differenthardware DSID 306 to each resource group 302, the graphics processor mayclassify setting the two resource groups 302 as a programming error andset the hardware DSID 306 for both resource groups 302 to a defaulthardware DSID 306.

FIG. 4 is an implementation of a command buffer 208 that includes one ormore set commands 408, one or more write commands 410, one or more readcommands 412, and one or more drop commands 414 that reference a createdresource group. Recall that a general-purpose processor (e.g., a CPU)presents and commits a command buffer 208 for execution on a graphicsprocessor. After the general-purpose processor commits the commandbuffer 208, the general-purpose processor is unable to encode additionalcommands into the command buffer 208. FIG. 4 depicts that a commandbuffer 208 includes set commands 408A and 408B, write commands 410A and410B, read command 412A, and drop command 414A that reference a createdresource group 302. The command buffer 208 includes three differentsections 402, 404, and 406. Section 402 represents commands that acommand encoder encodes to command buffer 208; section 404 representscommands that a different command encoder (e.g., a render commandencoder) encodes to command buffer 208; and section 406 representscommands that another command encoder encodes to command buffer 208.Each command encoder may be associated with specific graphics APIresources (e.g., buffers and textures) and states (e.g., stencil stateand pipeline state) for encoding the commands within each section 402,404, and 406 of command buffer 208.

With reference to FIG. 4, the set command 408A allows a developer and/orapplication to indicate that at least a portion of the command buffer208 will operate on a referenced resource group 302. Within sections 402and 404, the command buffer 208 includes a set command 408A that assignsa DSID (e.g., DSID #1) to a created resource group 302, a write command410A that writes data into the referenced resource group 302, a readcommand that reads data from the referenced resource group 302, and adrop command that drops the DSID associated with the resource group 302.The command encoder that encodes commands into section 402 tags thewrite command 410A with the allocated DSID (e.g., DSID #1) and anaddress range for the resource group 302. Subsequent command encodersalso inherit DSID state information and will tag the DSID (e.g., DSID#1) to commands referencing resource group 302. As an example, readcommand 412A is also tagged with the DSID #1 to identify resource group302. Having the read command 214A tagged with the allocated DSIDprotects against accessing dropped data because of hardware-levelsame-command buffer parallelism. Command encoders that encode commandsto command buffer 208 inherit the DSID state information until executingdrop command 414.

A command buffer 208 can include multiple write and reads from a createdresource group 302. In one implementation, after drop command 414, if anapplication and/or developer desires to write and read back into thesame resource group 302, the application and/or developer issues anotherAPI call to set the resource group 302 again. Using FIG. 4 as anexample, because of drop command 414A, the command encoder that encodescommands within section 406 does not inherit the DSID state that the setcommand 408A establishes. Instead, after drop command 414A, the commandencoder encodes a second set command 408B, which assigns a new DSID(e.g., DSID #2) to resource group 302. The command encoder uses the sameDSID state information that corresponds to the second set command 408Band tags the new DSID (e.g., DSID #2) with write command 410B.

FIG. 5 is an implementation of referencing a created resource groupacross multiple command buffers 208A, 208B, and 208C within a commandqueue 210. Multiple command buffers 208 may include DSID commands thatreference the same created resource group. In FIG. 5, because DSID stateinformation does not get inherited across command buffers, each commandbuffer 208 includes a set command 408 prior to implementing othercommands that reference the resource group. Similar to FIG. 4, at leasta portion of the command buffer inherits the DSID state informationassociated with set command 408. In comparison to FIG. 4, FIG. 5illustrates that the write command 410A, read command 412A, and dropcommand 414A are located on different command buffer 208A, 208B, and208C, respectively, rather than a single command buffer 208.

As shown in FIG. 5, each command buffer 208A, 208B, and 208C includesits own set command 408A, 408B, and 408C, respectively, since DSID stateinformation does not get inherited across command buffers 208.Specifically, within command buffer 208A, the set command 408A assigns aDSID (e.g., DSID #1) to the created resource group and the write command410A, which occurs after set command 408A, is tagged with the assignedDSID (e.g., DSID #1) for writing into the resource group. Command buffer208B, which a graphics processor subsequently executes after executingcommand buffer 208A, includes a set command 408B that assigns the DSID(e.g., DSID #1) to the created resource group. The read command 412A istagged with the assigned DSID (e.g., DSID #1) for writing into theresource group. Command buffer 208C, which the graphics processorexecutes after executing command buffer 208B, includes a set command408C that assigns the DSID (e.g., DSID #1) to the created resourcegroup. The drop command 414A references the assigned DSID (e.g., DSID#1) for providing a drop hint to the memory cache to delete contentswithin the cache line that corresponds to the assigned DSID.

FIG. 6 is an implementation of referencing a created resource groupacross multiple command queues 210. In comparison to FIG. 5, FIG. 6illustrates that the write command 410A is located in a command buffer208A within command queue 210A, and drop command 414A is located in adifferent command buffer 2080 in a different command queue 210B. Asshown in FIG. 6, each command buffer 208A and 2080 includes its own setcommand 408A and 408B since different command buffers do not inheritDSID state information. A command encoder places a fence update command602 after the write command 410A within command buffer 208A since theread command 412A is located in a different command queue 210. Incommand buffer 2080, a fence wait command 604 is placed before readcommand 412A and drop command 414A indicating that the write command410A in command queue 210A executes prior to executing the read command412A located in command queue 210B.

In FIG. 6, an application may insert a fence update command 602 andfence wait command 604 to track and manage resource dependencies acrosscommand queues 210. Resource dependencies arise as resources areproduced and consumed by different commands, regardless of whether thosecommands are encoded to the same command queue 210 or different commandqueues 210. The fence wait command 604 captures a graphics processor'swork up to a specific point in time. When the graphics processorencounters the fence wait command 604, the graphics processor waitsuntil relevant captured work is completed (e.g., receives acorresponding a fence update command 602) before continuing execution.

Although FIGS. 4-6 illustrate specific implementations of commandbuffers 208 that include DSID commands, the disclosure is not limited tothe specific implementation illustrated in FIGS. 4-6. For instance, eventhough FIG. 4 illustrates two set commands 408A and 408B, otherimplementations of command buffer 208 could include more than two setcommands 408 or single set command 408 that references resource group302. For FIG. 5, other implementations of command buffers 208A, 208B,and 208C could each include more than one set command 408, write command410, read command 412, or drop command 414. The use and discussion ofFIGS. 4-6 is only an example to facilitate ease of description andexplanation.

FIG. 7 depicts a flowchart illustrating a graphics processing operation700 for managing memory cache for graphics processing. To manage memorycache, operation 700 is able to generate DSID commands that reference aresource group within a single command buffer, across multiple commandbuffers, or across multiple command queues. In one implementation,operation 700 may be implemented by processor resource 110 shown inFIGS. 1 and 2. For example, blocks within operation 700 could beimplemented by the user space driver 102 and/or kernel driver 103 shownin FIG. 1. The use and discussion of FIG. 7 is only an example tofacilitate explanation and is not intended to limit the disclosure tothis specific example. As an example, block 702 may be optional suchthat operation 700 may not perform block 702 each time operation 700assigns, tags, and drops a DSID for a referenced resource group.

Operation 700 may start at block 702 and create a resource group.Operation is able to create a resource group using a variety operationsthat include, but are not limited to creating a new resource group,copying an existing resource group, or performing a mutable copy of anexisting resource group. Operation 700 then moves to block 704 andgenerates a set command that allocates a DSID to the created resourcegroup. Using FIGS. 4-6, the set command may occur before a writecommand, before a read command, and/or before a drop command. In one ormore implementations, operation 700 may have previously generated a setcommand in another command buffer (e.g., within the same command queueor different command queue). As previously discussed in FIG. 3,operation 700, based on the set command, may utilize a memoryvirtualization architecture to map the DSID to a hardware DSID. Having amemory virtualization architecture that maps DSID 304 to hardware DSID306 allows the graphics API architecture of managing and assigning DSID304 to be separate and independent from the hardware architecture formanaging and assigning hardware DSID 306.

At block 706, operation 700 generates, within a command buffer, a writecommand that references the DSID for writing to the resource group. Aspreviously discussed, at block 704, operation 700 assigns the DSID to aresource group. Based on the DSID assignment, if the write commandcauses at least some of the data for the resource group to be writteninto one or more cache lines, operation 700 associates the DSID to thecache lines. Operation 700 then moves block 708 and generates a readcommand referencing the DSID for reading from the resource group. Inimplementations where the read command is located on the same commandbuffer as the set command, the read command inherits DSID stateinformation from the set command. In instances where the read command islocated on a different command buffer, operation 700 may generate anadditional set command (not shown in FIG. 7) prior to generating theread command within the different command buffer. Additionally,operation 700 may generate a fence waiting command before the readcommand for situations where the write command generated in block 706 islocated in a different command queue than the read command generated inblock 708.

After completing block 708, operation 700 may then move to block 710 andgenerate a drop command that references the created DSID. The dropcommand generates a drop hint that informs memory cache to drop contentfrom a cache line that maps to the DSID. Stated another way, operation700 does not guarantee that the drop command causes the memory cache todrop content within the identified cache line, but instead notifies thememory cache that an application is done with the contents identified bythe DSID. For example, after memory cache receives a drop hint from theoperation 700, if memory cache still contains the content thatcorresponds to the DSID, then memory cache drops the cache line (e.g., adirty cache line) without flushing the contents to system memory.Alternatively, memory cache does not drop contents within the cache linethat corresponds to the DSID in situations where the memory cache hasalready persisted the cache line to system memory. Once, a graphicsprocessor completes executing the drop command, the DSID becomesavailable to reassign to a subsequent set command.

Illustrative Hardware and Software

The disclosure may have implication and use in and with respect tovariety of electronic devices, including single-and multi-processorcomputing systems, and vertical devices (e.g., cameras, gaming systems,appliances, etc.) that incorporate single-or multi-processing computingsystems. The discussion herein is made with reference to a commoncomputing configuration for many different electronic computing devices(e.g., computer, laptop, mobile devices, etc.). This common computingconfiguration may have a CPU resource including one or moremicroprocessors and a graphics processing resource including one or moreGPUs. Other computing systems having other known or common hardwareconfigurations (now or in the future) are fully contemplated andexpected. While the focus of some of the implementations relate tomobile systems employing minimized GPUs, the hardware configuration mayalso be found, for example, in a server, a workstation, a laptop, atablet, a desktop computer, a gaming platform (whether or not portable),a television, an entertainment system, a smart phone, a phone, or anyother computing device, whether mobile or stationary, vertical, orgeneral purpose.

Referring to FIG. 8, the disclosed implementations may be performed byrepresentative computing system 800. For example the representativecomputer system may act as an end-user device or any other device thatproduces or displays graphics. For example, computing system 800 may beembodied in electronic devices, such as a general purpose computersystem, a television, a set top box, a media player, a multi-mediaentertainment system, an image processing workstation, a hand-helddevice, or any device that may be coupled with or may incorporatedisplay or presentation devices as discussed herein. Computing system800 may include one or more processors 805, memory 810 (810A and 810B),one or more storage devices 815, and graphics hardware 820 (e.g.,including one or more graphics processors). Computing system 800 mayalso have device sensors 825, which may include one or more of: depthsensors (such as a depth camera), 3D depth sensor(s), imaging devices(such as a fixed and/or video-capable image capture unit), RGB sensors,proximity sensors, ambient light sensors, accelerometers, gyroscopes,any type of still or video camera, LIDAR devices, SONAR devices,microphones, CCDs (or other image sensors), infrared sensors,thermometers, etc. These and other sensors may work in combination withone or more GPUs, DSPs or conventional microprocessors along withappropriate programming so the sensor outputs may be properlyinterpreted and/or combined and interpreted.

Returning to FIG. 8, system 800 may also include communication interface830, user interface adapter 835, and display adapter 840—all of whichmay be coupled via system bus, backplane, fabric or network 845. Memory810 may include one or more different types of non-transitory media(e.g., solid-state, DRAM, optical, magnetic, etc.) used by processor 805and graphics hardware 820. For example, memory 810 may include memorycache, read-only memory (ROM), and/or random access memory (RAM).Storage 815 may include one or more non-transitory storage mediaincluding, for example, magnetic disks (fixed, floppy, and removable)and tape, optical media such as CD-ROMs and digital video disks (DVDs),and semiconductor memory devices such as Electrically ProgrammableRead-Only Memory (EPROM), solid state storage drives, and ElectricallyErasable Programmable Read-Only Memory (EEPROM). Memory 810 and storage815 may be used to retain media data (e.g., audio, image, and videofiles), preference information, device profile information, computerprogram instructions organized into one or more modules and written inany desired computer programming language, and any other suitable data.When executed by processor 805 and/or graphics hardware 820, suchcomputer program code may implement one or more of operations orprocesses described herein. In addition, the system may employmicrocontrollers (not shown), which may also execute such computerprogram code to implement one or more of the operations or computerreadable media claims illustrated herein. In some implementations, themicrocontroller(s) may operate as a companion to a graphics processor ora general-purpose processor resource.

Communication interface 830 may include semiconductor-based circuits andmay be used to connect computing system 800 to one or more networks.Illustrative networks include, but are not limited to: a local network,such as a USB network; a business's local area network; and a wide areanetwork such as the Internet and may use any suitable technology (e.g.,wired or wireless). Communications technologies that may be implementedinclude cell-based communications (e.g., LTE, CDMA, GSM, HSDPA, etc.) orother communications (Apple lightning, Ethernet, WiFi®, Bluetooth®, USB,Thunderbolt®, Firewire®, etc.). (WIFI is a registered trademark of theWi-Fi Alliance Corporation. BLUETOOTH is a registered trademark ofBluetooth Sig, Inc. THUNDERBOLT and FIREWIRE are registered trademarksof Apple Inc.). User interface adapter 835 may be used to connectkeyboard 850, microphone 855, pointer device 860, speaker 865, and otheruser interface devices such as a touchpad and/or a touch screen (notshown). Display adapter 840 may be used to connect one or more displays870.

Processor 805 may execute instructions necessary to carry out or controlthe operation of many functions performed by computing system 800 (e.g.,evaluation, transformation, mathematical computation, or compilation ofgraphics programs, etc.). Processor 805 may, for instance, drive display870 and receive user input from user interface adapter 835 or any otheruser interfaces embodied by a system. User interface adapter 835, forexample, can take a variety of forms, such as a button, a keypad, atouchpad, a mouse, a dial, a click wheel, a keyboard, a display screen,and/or a touch screen. In addition, processor 805 may be based onreduced instruction-set computer (RISC) or complex instruction-setcomputer (CISC) architectures or any other suitable architecture and mayinclude one or more processing cores. Graphics hardware 820 may bespecial purpose computational hardware for processing graphics and/orassisting processor 805 in performing computational tasks. In someimplementations, graphics hardware 820 may include CPU-integratedgraphics and/or one or more discrete programmable GPUs. Computing system800 (implementing one or more implementations discussed herein) canallow for one or more users to control the same system (e.g., computingsystem 800) or another system (e.g., another computer or entertainmentsystem) through user activity, which may include audio instructions,natural activity, and/or pre-determined gestures such as hand gestures.

Various implementations within the disclosure may employ sensors, suchas cameras. Cameras and like sensor systems may include auto-focussystems to accurately capture video or image data ultimately used in avariety of applications, such as photo applications, augmented realityapplications, virtual reality applications, and gaming. Processingimages and performing recognition on the images received through camerasensors (or otherwise) may be performed locally on the host device or incombination with network accessible resources (e.g., cloud serversaccessed over the Internet).

Returning to FIG. 8, device sensors 825 may capture contextual and/orenvironmental phenomena such as time; location information; the statusof the device with respect to light, gravity, and the magnetic north;and even still and video images. In addition, network-accessibleinformation such as weather information may also be used as part of thecontext. All captured contextual and environmental phenomena may be usedto provide context to user activity or information about user activity.For example, in accessing a gesture or the expression or emotion of auser, the contextual information may be used as part of the analysis,and the analysis may be performed using the techniques discussed herein.

Output from the device sensors 825 may be processed, at least in part,by processors 805 and/or graphics hardware 820, and/or a dedicated imageprocessing unit incorporated within or without computing system 800.Information so captured may be stored in memory 810 and/or storage 815and/or any storage accessible on an attached network. Memory 810 mayinclude one or more different types of media used by processor 805,graphics hardware 820, and device sensors 825 to perform devicefunctions. Storage 815 may store data such as media (e.g., audio, image,and video files); metadata for media; computer program instructions;graphics programming instructions and graphics resources; and othersoftware, including database applications (e.g., a database storingavatar frames), preference information, device profile information, andany other suitable data. Memory 810 and storage 815 may be used toretain computer program instructions or code organized into one or moremodules in either compiled form or written in any desired computerprogramming language. When executed by, for example, a microcontroller,GPU or processor 805, such computer program code may implement one ormore of the acts or functions described herein (e.g., interpreting andresponding to user activity including commands and/or gestures).

As noted above, implementations within this disclosure include software.As such, a description of common computing software architecture isprovided as expressed in a layer diagram in FIG. 9. Like the hardwareexamples, the software architecture discussed here is not intended to beexclusive in any way, but rather to be illustrative. This is especiallytrue for layer-type diagrams, which software developers tend to expressin somewhat differing ways. In this case, the description begins withlayers starting with the base hardware layer 995 illustrating hardwarelayer 940, which may include memory, general purpose processors,graphics processors, microcontrollers, or other processing and/orcomputer hardware such as memory controllers and specialized hardware.Above the hardware layer is the operating system kernel layer 990showing an example as operating system kernel 945, which is kernelsoftware that may perform memory management, device management, andsystem calls. The operating system kernel layer 990 is the typicallocation of hardware drivers, such as a graphics processor drivers. Thenotation employed here is generally intended to imply that softwareelements shown in a layer use resources from the layers below andprovide services to layers above. However, in practice, all componentsof a particular software element may not behave entirely in that manner.

Returning to FIG. 9, operating system services layer 985 is exemplifiedby operating system services 950. Operating system services 950 mayprovide core operating system functions in a protected environment. Inaddition, operating system services shown in operating system serviceslayer 985 may include frameworks for OpenGL®/OpenCL® 951, CUDA® or thelike, Metal® 952, user space drivers 953, and a Software Rasterizer 954.(OPENGL is a registered trademark of Silicon Graphics InternationalCorporation. OPENCL is a registered trademark of Apple Inc. CUDA is aregistered trademark of NVIDIA Corporation.) While most of theseexamples all relate to graphics processor processing or graphics and/orgraphics libraries, other types of services are contemplated by varyingimplementations of the disclosure. These particular examples alsorepresent graphics frameworks/libraries that may operate in the lowertier of frameworks, such that developers may use shading and primitivesand/or obtain fairly tightly coupled control over the graphics hardware.In addition, the particular examples named in FIG. 9 may also pass theirwork product on to hardware or hardware drivers, such as the graphicsprocessor driver, for display-related material or compute operations.

Referring again to FIG. 9, OpenGL®/OpenCL® 951 represent examples ofwell-known libraries and application programming interfaces for graphicsprocessor compute operations and graphics rendering including 2D and 3Dgraphics. Metal® 952 also represents a published graphics library andframework, but it is generally considered lower level than OpenGL/OpenCL951, supporting fine-grained, low-level control of the organization,processing, and submission of graphics and computation commands, as wellas the management of associated data and resources for those commands.User space drivers 953 is software relating to the control of hardwarethat exists in the user space for reasons that are typically related tothe particular device or function. In many implementations, user spacedrivers 953 work cooperatively with kernel drivers and/or firmware toperform the overall function of a hardware driver. Software Rasterizer954 refers generally to software used to make graphics information suchas pixels without specialized graphics hardware (e.g., using only theCPU). These libraries or frameworks shown within the operating systemservices layer 985 are only exemplary and intended to show the generallevel of the layer and how it relates to other software in a samplearrangement (e.g., kernel operations usually below and higher-levelapplications services 960 usually above). In addition, it may be usefulto note that Metal® 952 represents a published framework/library ofApple Inc. that is known to developers in the art. Furthermore,OpenGL®/OpenCL® 951 may represent frameworks/libraries present incurrent versions of software distributed by Apple Inc.

Above the operating system services layer 985 there is an applicationservices layer 980, which includes Sprite Kit 961, Scene Kit 962, CoreAnimation 963, Core Graphics 964, and other applications services 960.The operating system services layer 985 represents higher-levelframeworks that are commonly directly accessed by application programs.In some implementations of this disclosure the operating system serviceslayer 985 includes graphics-related frameworks that are high level inthat they are agnostic to the underlying graphics libraries (such asthose discussed with respect to operating system services layer 985). Insuch implementations, these higher-level graphics frameworks are meantto provide developer access to graphics functionality in a moreuser/developer friendly way and allow developers to avoid work withshading and primitives. By way of example, Sprite Kit 961 is a graphicsrendering and animation infrastructure made available by Apple Inc.Sprite Kit 961 may be used to animate textured images or “sprites.”Scene Kit 962 is a 3D-rendering framework from Apple Inc. that supportsthe import, manipulation, and rendering of 3D assets at a higher levelthan frameworks having similar capabilities, such as OpenGL. CoreAnimation 963 is a graphics rendering and animation infrastructure madeavailable from Apple Inc. Core Animation 963 may be used to animateviews and other visual elements of an application. Core Graphics 964 isa two-dimensional drawing engine from Apple Inc., which provides 2Drendering for applications.

Above the application services layer 980, there is the application layer975, which may comprise any type of application program. By way ofexample, FIG. 9 shows three specific applications: photos 971 (a photomanagement, editing, and sharing program), Quicken® 972 (a financialmanagement program), and iMovie® 973 (a movie making and sharingprogram). (QUICKEN is a registered trademark of Intuit Inc. IMOVIE is aregistered trademark of Apple Inc.). Application layer 975 also showstwo generic applications 970 and 974, which represent the presence ofany other applications that may interact with or be part of theinventive implementations disclosed herein. Generally, someimplementations of the disclosure employ and/or interact withapplications that produce displayable and/or viewable content or producecomputational operations that are suited for GPU processing.

In evaluating operating system services layer 985 and applicationsservices layer 980, it may be useful to realize that differentframeworks have higher- or lower-level application program interfaces,even if the frameworks are represented in the same layer of the FIG. 9diagram. The illustration of FIG. 9 serves to provide a generalguideline and to introduce exemplary frameworks that may be discussedlater. Furthermore, some implementations of the disclosure may implythat frameworks in application services layer 980 make use of thelibraries represented in operating system services layer 985. Thus, FIG.9 provides intellectual reinforcement for these examples. Importantly,FIG. 9 is not intended to limit the types of frameworks or librariesthat may be used in any particular way or in any particularimplementation. Generally, many implementations of this disclosurerelate to the ability of applications in layer 975 or frameworks inlayers 980 or 985 to divide long continuous graphics processor tasksinto smaller pieces. In addition, many implementations of the disclosurerelate to graphics processor (e.g., GPU) driver software in operatingsystem kernel layer 990 and/or embodied as microcontroller firmware inhardware layer 995; such drivers performing a scheduling function forthe graphics processor resource (e.g., GPU).

FIG. 10 illustrates a software architecture similar to the standardarchitecture shown in FIG. 9. By way of distinction, the architecture ofFIG. 10 shows: user space graphics drivers 1005A and 1005B; kernelgraphics drivers 1010A and 1010B in the operating system kernel 945; amicrocontroller 1015, accompanied by microcontroller firmware 1020,including graphics driver firmware 1025 in the hardware layer 940; andexecution cores 1030 in the hardware layer 940. The presence of multipleinstances of a graphics driver (user space graphics drivers 1005A and1005B, kernel graphics drivers 1010A and 1010B, and graphics driverfirmware 1025 in the microcontroller firmware 1020) indicates thevarious options for implementing the graphics driver. As a matter oftechnical possibility any of the three shown drivers might independentlyoperate as a sole graphics driver. In some implementations of thedisclosure, the overall graphics driver is implemented in a combinationof kernel graphics drivers 1010A and 1010B and graphics driver firmware1025 (e.g., in the operating system kernel 945 and the microcontrollerfirmware 1020, respectively). In other implementations, the overallgraphics driver may be implemented by the combined effort of all threeshown drivers 1005A and 1005B, 1010A and 1010B, and 1025.

At least one implementation is disclosed and variations, combinations,and/or modifications of the implementation(s) and/or features of theimplementation(s) made by a person having ordinary skill in the art arewithin the scope of the disclosure. Alternative implementations thatresult from combining, integrating, and/or omitting features of theimplementation(s) are also within the scope of the disclosure. Wherenumerical ranges or limitations are expressly stated, such expressranges or limitations may be understood to include iterative ranges orlimitations of like magnitude falling within the expressly stated rangesor limitations (e.g., from about 1 to about 10 includes, 2, 3, 4, etc.;greater than 0.10 includes 0.11, 0.12, 0.13, etc.). The use of the term“about” means ±10% of the subsequent number, unless otherwise stated.

Many other implementations will be apparent to those of skill in the artupon reviewing the above description. The scope of the inventiontherefore should be determined with reference to the appended claims,along with the full scope of equivalents to which such claims areentitled. In the appended claims, the terms “including” and “in which”are used as the plain-English equivalents of the respective terms“comprising” and “wherein.”

What is claimed is:
 1. A non-transitory program storage device, readableby a processor and comprising instructions stored thereon to cause theprocessor to: create a resource group for a plurality of graphicsapplication program interface (API) resources, wherein each graphics APIresource corresponds to a memory allocation for storing data that isaccessible to a graphics processor; encode a set command that referencesthe resource group within a command buffer, wherein the set commandcauses a data set identifier (DSID) to be associated with the resourcegroup encode a write command that references the resource group withinthe command buffer that causes the graphics processor to write data intoa cache line within a memory cache, wherein the write command causes thecache line to be associated with the DSID; encode a drop command thatcauses the graphics processor to notify the memory cache to drop,without flushing to memory, data stored within the cache line; andcommit one or more command buffers that include the set command, thewrite command and the drop command for execution on the graphicsprocessor.
 2. The non-transitory program storage device of claim 1,wherein the instructions further cause the processor to encode a readcommand that causes the graphics processor to read data written into theresource group.
 3. The non-transitory program storage device of claim 2,wherein the write command and the read command are encoded withindifferent command buffers.
 4. The non-transitory program storage deviceof claim 2, wherein the write command and the read command are encodedwithin a single command buffer.
 5. The non-transitory program storagedevice of claim 2, wherein the instructions further cause the processorto: encode a second set command within a second command buffer of theone or more command buffers; and encode the read command within thesecond command buffer.
 6. The non-transitory program storage device ofclaim 1, wherein the drop command is encoded within a second commandbuffer.
 7. The non-transitory program storage device of claim 6, whereinthe command buffer is sorted within a first queue and the second commandbuffer is sorted within a second queue.
 8. The non-transitory programstorage device of claim 7, wherein the instructions further cause theprocessor to: encode a fence update command after the write commandwithin the command buffer in the first queue; and encode a fence waitcommand before the drop command within the second command buffer in thesecond queue.
 9. The non-transitory program storage device of claim 1,wherein the drop command is associated with the DSID that references theresource group, and wherein the drop command causes the graphicsprocessor to notify the memory cache to drop data stored within thecache line associated with the DSID.
 10. A system comprising: memory;and a processor operable to interact with the memory, and configured to:encode a set command that references a resource group within a commandbuffer, wherein the set command causes a data set identifier (DSID) tobe associated with the resource group that includes a plurality ofgraphics application program interface (API) resources; encode a writecommand that references the resource group within the command bufferthat causes a graphics processor to write data into a cache line withina memory cache, wherein the write command causes the DSID to be taggedwith the cache line; encode a drop command that references the DSID andcauses the graphics processor to notify the memory cache to delete,without evicting to memory, data stored within the cache line; andcommit one or more command buffers that include the set command, thewrite command, and the drop command for execution on the graphicsprocessor.
 11. The system of claim 10, wherein the processor is furtherconfigured to: encode a read command that causes the graphics processorto read data written into the resource group.
 12. The system of claim11, wherein the drop command is associated with the DSID that referencesthe resource group, and wherein the drop command causes the cache linetagged with the DSID to be dropped.
 13. The system of claim 10, whereinthe processor is further configured to: encode a second set commandwithin a second command buffer of the one or more command buffers; andencode a read command within the second command buffer, wherein the readcommand causes the graphics processor to read data written into theresource group.
 14. The system of claim 13, wherein the drop command isencoded within the second command buffer.
 15. The system of claim 13,wherein the command buffer is sorted within a first queue and the secondcommand buffer is sorted within a second queue.
 16. The system of claim15, wherein the processor is further configured to: encode a fenceupdate command after the write command within the command buffer in thefirst queue; and encode a fence wait command before the drop commandwithin the second command buffer in the second queue.
 17. The system ofclaim 10, wherein the processor is further configured to: reuse the DSIDafter the graphics processor completes the execution of the dropcommand.
 18. A non-transitory program storage device, readable by aprocessor and comprising instructions stored thereon to cause theprocessor to: obtain from one or more command buffers that includes awrite command and a drop command that are associated with a data setidentifier (DSID), wherein the DSID is associated with a resource groupthat includes a plurality of graphics application program interface(API) resources and a cache line in a memory cache; execute the writecommand to write data within the cache line to generate a dirty cacheline, wherein the write command causes the cache line to be associatedwith the DSID; and execute, after the write command, the drop command tonotify the memory cache to drop the written data within the dirty cacheline, wherein the written data is dropped without flushing the dirtycache line to main memory.
 19. The non-transitory program storage deviceof claim 18, wherein the processor is a graphics processing unit (GPU)and the command buffers are encoded by a central processing unit (CPU).20. The non-transitory program storage device of claim 18, wherein thedrop command is associated with the DSID that references the resourcegroup, and wherein the drop command causes the cache line associatedwith the DSID to be dropped.